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If humans are born equal or can become equal with regard 
to learning, then the home and the school have 
responsibilities far greater than they have assumed in 
the past. (Benjamin B1oom«1976} 



-Abstract- 



Entrance and exit examinations play a very important role in higher 
education in the United States. For many the entrance examination has been 
used to exclude them from the world of higher education. As a result, we 
have prevented a portion of our population from traveling the road of 
••upward mobility" upon which many of our grandparents traveled. The exit 
examination is meant to evaluate those who have had the opportunity to 
travel that road, but in too many cases the examination and the procedures^ 
surrounding it are of questio»iable integrity. 



This paper centers on the Quality Assurance Model, commonly referred 
to as the Course Development Model (the Model), and describes how it can be 
used to foster good test development and improve Instruction in 
Institutions of higher education. The Course Development Model as first 
discussed at the Testing and Quality Assurance in Higher Education 
Conference In Miami In February 1986 showed how the Model can develop 
faculty accountability as well as being used as one Item In the faculty 



evaluation process. 
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appears, at UCF, are already following portions of the Model. We only have 
to close the loop. If we do the result will be better examinations and 
improved instruction. 

INTRODUCTION 

In his opening remarks to the Testing »nd Quality Assurance in Higher 
Education Conference in Miami, Florida om 12 February 1986, Dr. Robert McCabe, 
the President of Miami-Dade Community College, told the assemblage that 
assessment was a growing issue with both state legislatures and the public. 
There is, he said, a mistrust of what educators are doing. We as educators must 
ask ourselves the question: "Do wr really know what we are doing?" Our goal 
should be the assurance that more people are achieving higher competencies. 
Assessment then, according to Dr. McCabe, is of enormous importance. Assessment 
can and must have a positive impact on programs, curricula, and instruction. 

Kenneth B. Clarke, the keynote speaker, followed Dr. McCabe at that same 
conference. He said that part of the problem was in the concept of diagnostic 
versus rejective use of tests and measurements. We have not explored this 
relationship, nor have we expounded on it, nor have we integrated the dif- 
ferences between them in our educational system. In order to increase the 
opportunities of higher education for all our citizens, improve the instruc- 
tional process, and dev3lop diagnostic measures to evaluate learning we must 
review and analyze the relationship between curriculum and instruction and find 
methods for strengthening this association. 

COURSE DEVELOPMENT MODEL 

There are many reasons for tightening this relationship between curriculum 
and instruction. The first is to improve accountability which is one of the 



growing issues in higher education. A second is to develop better testing 
instruments. The third reason is to improve the instructional process itself. 

Glenny and Schmidtlein (1983) state that state legislatures and depart- 
ments of higher education will scrutinize the budgets for state supported colle- 
ges and universities more closely as the student population continues to decline 
and as the availability of funds becomes more scarce. They believe thet one 
method of control will be intrusion into the educational process itself. The 
most likely areas of this invasion will be to develop criteria for the eva- 
luation of new and existing academic programs and faculty effectiveness. A clo- 
sely related area will be the oversight of exit evaluations of student prepared- 
ness to demonstrate competence in their fields of specialization. 

How can educators contribute to this inevitable process to preserve 
academic freedom, protect the interests of students, and at the same time 
provide legislatures and the citizenry as a whole the assurance they want. The 
Course Development Model meets all of these objectives and provides a high 
degree of accountability discussed in this author's presentation to the Quality 
Assurance in Higher Education Conference in February 1986. The Model, depicted 
in figure 1, requires that course descriptions contained in college and univer- 
sity catalogues sufficiently describe the course content in concrete terms that 
would enable an instructor to derive sound course objectives from them. The 
course objectives can then be used to develop the tests and measuring devices 
used to evaluate student performance and learning. It is important to note, 
however, that the model requires that a test construction taxonomy, such as 
Bloom's, be used as a guide to develop these tests. The final step is the 
evaluation of tests and measuring instruments for reliability and validity. The 



cycle is completed only when the evaluation of the tests leads to improvement 
of the instructional process- 
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The obvious question at this juncture is why do we need such a model? The 
intrusion of state legislatures, boards of trustees of private colleges and uni- 
versities, and the public at large are not sufficient justification for this 
additional work which will be required as the benefits will be minimal. Quite 
the contrary! 

When we review the various theories for development of curriculum we 
notice that the theorists have not provided us with the means to get to the 
instructional process. When we review the theories of instruction we are not 
provided with the how to base the instruction on the information provided by the 
curriculum. There is a gap which clearly emerges when we view these two side- 



by-side. The Course Development Model provides the administrator, faculty 
member, and evaluator with a clear, easy to follow procedure and audit trail 
leading from the curriculum to. the instruction. This model does not impact on 
the way the curriculum is developed, nor does it infringe on the faculty member 
and his or her method of instruction. It does, however, insure that the two are 
well connected and interrelated. 

PURPOSE AND BACKGROUND 

Dr. Clifford Aldeman of the National Institute of Education says that 
college faculty are rot trained in the design, development, and evaluation of 
tests. If they were trained in these areas the instructional process would be 
improved. His statement suggests that there is a need for some kind of faculty 
development program. 

Elbe (1972) suggests that faculty development should be an integral part 

of all institutions of higher education. The Course Development Model provides 

a tool for helping an institution identify areas for faculty development. 

Based on Gaff's (1975) findings the input factor of the model is important. 

"The kinds or change that emanate from instructional 
improvement programs are more in the areas of process, 
teaching methodology and techniques, learning materials, 
and interpersonal relationships. And the changes that 
do occur will probably have more impact directly on 
individuals and small groups than indirectly through 
changes in organizations as a whole" (p. 164). 

Of the areas Gaff identified as needing attention, the Model will successfully 

impact on design of new courses, redesign of existing courses, and improvement 

of instructional materials. 

Carlberg (1981) states that about 15% of the instructional time available 
to teachers is devoted to testing activities. Therefore, it is essential, that 



that faculty know how to develop good measuring devices. If we assume that 
Dr. Adelman is correct then one major area for faculty development is in the 
area of test development. Carlberg also indicates that careful consideration of 
testing is important because of the impact testing has on the planning of 
instructional strategies. 

Hopkins and Stanley (1981) state that research has shown that tests can 
improve student learning. As a resu'it, measurement specialists have identified 
a large number of strategies for employing tests in the classroom. Goslin 
(1967) supports the idea that faculty have instruction on how to develop high 
quality instruments. The Bloom Taxonomy provides the faculty member with a 
proven methodology and process for test design, and development. 

Knapper (1979) states that effective instruction is closely linked to 
effective evaluation. The problem, according to Knapper, is few instructors 
evaluate the way instruction is presented. He also notes that teaching prac- 
tices and learning effectiveness can be substantially changed if the evaluation 
process is conducted properly. Instructors must be willing, something which all 
too many are unwilling to do, to evaluate their current teaching practices. The 
Course Development Model forces faculty members to evaluate the way in which 
they teach their students. 

Bloom (1976) says that most students can learn what most schools have to 
teach if the subject matter is approached in a systematic and sensitive fashion. 
Variations in levels of learning by students is based not only on the student's 
learning history, but also by the quality of instruction they receive. "In 
general » we can estimate that quality of instruction can account for at least 
one fourth (r=+.50) of the variance on relevant cognitive achievement measures" 
(p- 135). 

ERIC 



Thus, it is incumbant upon educators to design, develop, model, and use 
a systematic sensible approach to linking curriculum with instruction in order 
to transit the gap between the' two. Only by this linkage can we accurately and 
fairly evaluate students, and improve our instruction. The Course Development 
Model provides the tools necessary to accomplish these tasks. Do faculty use 
this or a similar model now, or is such a suggestion simply a theoretical 
fantasy? To answer this question a pilot study was conducted at the University 
of Central Florida in the Spring of 1985. The results of that study suggest 
that this model is realistic, feasible, and could be readily implemented with 
minimal behavior changes required on the part of the teaching faculty. 

ANALYSIS OF THE DATA 

This discussion of the research is the second of a two part analysis of 
the data obtained by this pilot project. The first part of the analysis is 
contained in a previously published paper, "A Quality Assurance Model for Higher 
Education", and is available through the Educational Resources Information 
Center (ERIC). The second part analyzes the data as it pertains to testing and 
improvement of instruction. 

All four hundred five members of the teaching faculty at the University of 
Central Florida were requested to particpate in this study by completing a sur- 
vey on the use of statistics by college faculty. One hundred and thirty-one 
completed surveys were returned. This represents a thirty-one percent return 
rate. Although this sample is biased, it does represent one third of the popu- 
lation, thus it is assumed to accurately represent the attitudes and behaviors 
of the UCF faculty. The breakdown of returned surveys by colleges within the 
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university is ns follows: Arts and Sciences - 44%, Business Administration - 
18*, College of Education - 21X, College of Engineering - 12*, and College of 
Health Sciences - 5*. 

The first statistical procedure to be performed was a frquency distribu- 
tion which provided the moments of the distribution and other statistical infor- 
mation. Table One contains this data for the review of the reader. For those 
cases when the between group significance was significant at the 0.5 leyel it is 
so indicated. I would like to call to the reader's attention that there is a 
significance between groups in the areas of "using course objectives to develop 
tests" and "using a taxonomy for test development". These are brought to your 
attention because the Course Development Model stresses their importance. 

An important element of the model is the generation and use of teacher 
made tests by faculty in the evaluation of student knowledge gained in the 
classroom. Over ninety-one per cent of the faculty design and develop all of 
the tests and measuring devices they use to evaluate student performance. The 
mean was 97.672. What makes this of great concern to us as educators is the 
statement by Dr. Aldeman concerning the need for faculty training or test proce- 
dures. The question in the survey which asks about the use of a taxonomy had a 
mean score of 3.33. This substantiates Dr. Aldeman's position. Further 
research will help determine whether or not this is unique to UCF or typical of 
higher education. I believe the latter to be correct. 

The number of tests developed for each course was found to be 4.12. The 
average number of tests actually given in each course averaged 4.07. The dif- 
ference in these two is assumed to be a result of rounding. The median and mode 
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TABLE 1 
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for each is 4.00. The Pearson Correlation between the two scores was .72 and 
significant to the .05 level. 

In order to look at the relationship of the variables comprising the 
Model, correlations between each were calculated. Table 2 shows the results of 
this investigation. The individual relationships are all significant except the 
correlation between using objectives to develop tests and obtaining reliability 
tests and measuring devices by faculty. Thus we can conclude that faculty 
indicate a desire to bring curriculum and instruction closer together but lack 
the Model to do so. 



MODEL: CORRELATION MATRIX* 
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Three separate three variable log linear analyses were conducted to test 
the strength of the relationships between selected variables. The first con- 
cerned, (1) using course descriptions to develop course objectives, (2) using 
the course objectives to develop tests, and (3) using a taxonomy, such as 
Bloom's, to design the testing instrument. The second was (1) using course 
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to be entertained. Adoption of the null model is analogus to performing a 
three-way ANOVA and discovering that, of the seven omnibus F tests, not one has 
achieved statisitcal significance" (Kennedy, p. 166). 



Hierarchical Logit Models for Four-Way Tables Where D Is the Logit Variable 

Logit 
Hodel 

No. Log-Llnear Logit Models Marginals Fitted 

(0) * X ♦a+b*c+d+ab*ac+bc+abc ABC,D 

(1) X ♦a+b*c+d+ab+ac+bc+abc+ad ABC, AD 
(Z) X ♦a+b*c+d+ab+ac+bc+abc+ad+bd ABC,AD,BD 

( 3 ) X ♦a+b*c+d+ab+ac+bc+abc+ad+bd+cd ABC , AD , BD , CD 

(4) X ♦a+b+c+d+ab+ac+bc+abc+ad+bd+cd+abd ABC,ABD,CD 

(5) X ♦a+b+c+d+ab+dc+bc+abc+ad+bd+cd+abd+acd ABC,ABD,ACD 

(6) 31^ ♦a+b*c+d+ab+ac+bc+abc+ad+bd+cd+abd+acd+bcd ABC,ABD,ACD,BCD 

(7) X ♦a+b*c+d+ab+ac+bc+abc+ad+bd+cd+abd+acd+bcd+abcd ABCD 



Note. Constituent terms (parameter estimates) in models are denoted by 
their respective superscripts with the exception of the first term, 

^Numbers assigned to logit models in this table no longer reflect the 
number of constituent terms. They do, however, reflect the number of interac- 
tion terms that Involve the logit variable. (Kennedy, p. 164) 

a « DESCR b - OBJ c « BLOOM d « ORELI 



TABLE 4 



At this point something must be said about the log linear analysis. A 
four variable analysis requires a great deal of computer memory. The running of 
this particular study literally shut down the computer center for over a half 
hour. No other programs were run during the time that the computer made the 
calculations for this analysis. Later a stepwise regression analysis was run 
which provided more tangible interpretations of the data. The other factor 
worth noting is that with a sample size of only one hundred thirty-one (131) 
meant that a large number of cells were empty. 



Changing the dependent variable to "using course descriptions to develop 
course objectives", a stepwise -multiple regression reveals that both the "use of 
course objectives to develop tests" and "use of a taxonomy, Bloom for instance", 
does correlate well with the course use of course descriptions variable. The 
correlation is statistically significant (r=.43). The two indicies of reliabi- 
lity were unimportant. As a result, we can say that faculty members who use 
course objectives to develop tests and use a taxonomy for test construction are 
in all likelihood going to use the course descriptions to develop their course 
objectives. The problem is we can only account for 19% of the variance which is 
not encouraging. 

The last two statistical procedures used in this study are a multiple 
regression and a stepwise multiple regression. Analysis of the data shows that 
there is a correlation among the four variables being considered. The step- 
wise multiple regression shows exactly what this correlation looks like. The 
R squared value is .1. The stepwise multiple regression reveals that the rela- 
tionship of using a taxonomy, like Bloom's, on reliability is only .08 
(R squared value). Thus, the other two independent variables have no impact at 
all and the impact of Bloom is Tinor although it is statistically significant. 

FACULTY DEVELOPMENT 

Analysis of the data leads to some very interesting interprteations which 
can have a strong impact on testing and improvement of instruction in higher 
education. At the present time faculty members do not have a theoretical model 
which enables them to move from curriculum to instruction in a logical, sequen- 
tial, and auditable fashion (figure 2). Yet, behaviorally, they exhibit some of 
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the behaviors called for in the Course Development Model, What is needed is 
instruction on how to bridge the gap between the curriculum and the professor *s 
preferred method(s) of instruction in the classroom. 




Figure 2 

For the most part, the behavioral changes are minor. Many faculty members 
already use the course descriptions contained in university and college catalo- 
gues to develop their course objectives. Both test development and instruction 
can be improved if college faculty will consistantly perform this task. For the 
most part it is a simple documentation procedure. Next, faculty members must 
consistantly use their course objectives to develop the tests and other 
measuring devices used in their classes. By using course objectives the tests 
should have content validity because of the process by which they are developed. 
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Faculty members must also be informed about taxonomies of educational 
objectives such as that developed by Benjamin Bloom and his associates. By 
using a taxonomy the tests which are used will be properly targeted to a domain 
of learning. In addition to the logical sequence of steps used and the audit 
trail they provide, the tests will have construct validity, since the tests 
will have been developed to test the construct. This also will significantly 
improve the the tests used in the classroom. 

Finally, by teaching faculty how to obtain reliability information on 
their tests and use this information the entire course development process will 
be improved. The reliability information can be used to not only improve the 
tests themselves, but to improve instruction by having the professor examine 
whether or not the instruction meets the requirements of the learner. If pro- 
perly schooled, the instructor can use the reliability information to improve 
the course descriptions and or objectives themselves. 

CONCLUSION 

But, why should college faculty adopt such a model? There are several 
reasons. First, the Model makes good pedagogical sense. Second, the public is 
demanding accountability. This Model provides for that accountability without 
requiring the faculty member to give up the independence now enjoyed. In 
addition, the Model does not require the writing of a new curriculum, nor does 
it mandate a change in the instructional practices now employed. Third, use of 
the Model will not only improve tests and other forms of evaluating student per- 
formance, but it will also help the professor improve instruction. Lastly, this 
model satisfies the state legislatures* desire for accountability and evaluation 
of faculty, provides a methodology for improving tests and instruction, yet pre- 
serves faculty independence. 
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